Light sensitive display with pressure sensor

ABSTRACT

A system comprising a display having a stack of layers configured to display an image, multiple elements included in the stack of layers and a pressure sensor. Each of the multiple elements is configured to sense light. The pressure sensor is distinct from the multiple elements and is configured to sense a pressure applied to the display.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.11/351,098, filed Feb. 8, 2006, which is a continuation of U.S. patentapplication Ser. No. 10/739,455, filed Dec. 17, 2003, the entireties ofwhich are incorporated herein by reference. This application is acontinuation-in-part of U.S. patent application Ser. No. 11/901,649,filed Sep. 18, 2007, which is a continuation of U.S. patent applicationSer. No. 10/371,413, filed Feb. 20, 2003, which claims the benefit ofU.S. Provisional Application No. 60/383,040, filed May 23, 2002, andU.S. Provisional Application No. 60/359,263, filed Feb. 20, 2002, theentireties of which are incorporated herein by reference.

BACKGROUND

The present invention relates to touch sensitive displays.

Touch sensitive screens (“touch screens”) are devices that typicallymount over a display such as a cathode ray tube. With a touch screen, auser can select from options displayed on the display's viewing surfaceby touching the surface adjacent to the desired option, or, in somedesigns, touching the option directly. Common techniques employed inthese devices for detecting the location of a touch include mechanicalbuttons, crossed beams of infrared light, acoustic surface waves,capacitance sensing, and resistive sensing techniques.

For example, Kasday, U.S. Pat. No. 4,484,179 discloses anoptically-based touch screen comprising a flexible clear membranesupported above a glass screen whose edges are fitted with photodiodes.When the membrane is flexed into contact with the screen by a touch,light which previously would have passed through the membrane and glassscreen is trapped between the screen surfaces by total internalreflection. This trapped light travels to the edge of the glass screenwhere it is detected by the photodiodes which produce a correspondingoutput signal. The touch position is determined by coordinating theposition of the CRT raster beam with the timing of the output signalsfrom the several photodiodes. The optically-based touch screen increasesthe expense of the display, and increases the complexity of the display.

Denlinger, U.S. Pat. No. 4,782,328 on the other hand, relies onreflection of ambient light from the actual touch source, such as afinger or pointer, into a pair of photosensors mounted at corners of thetouch screen. By measuring the intensity of the reflected light receivedby each photosensor, a computer calculates the location of the touchsource with reference to the screen. The inclusion of the photosensorsand associated computer increases the expense of the display, andincreases the complexity of the display.

May, U.S. Pat. No. 5,105,186, discloses a liquid crystal touch screenthat includes an upper glass sheet and a lower glass sheet separated byspacers. Sandwiched between the glass sheets is a thin layer of liquidcrystal material. The inner surface of each piece of glass is coatedwith a transparent, conductive layer of metal oxide. Affixed to theouter surface of the upper glass sheet is an upper polarizer whichcomprises the display's viewing surface. Affixed to the outer surface ofglass sheet is a lower polarizer. Forming the back surface of the liquidcrystal display is a transflector adjacent to the lower polarizer. Atransflector transmits some of the light striking its surface andreflects some light. Adjacent to transflector is a light detecting arrayof light dependent resistors whose resistance varies with the intensityof light detected. The resistance increases as the light intensitydecreases, such as occurs when a shadow is cast on the viewing surface.The light detecting array detects a change in the light transmittedthrough the transflector caused by a touching of viewing surface.Similar to touch sensitive structures affixed to the front of the liquidcrystal stack, the light sensitive material affixed to the rear of theliquid crystal stack similarly pose potential problems limiting contrastof the display, increasing the expense of the display, and increasingthe complexity of the display.

Touch screens that have a transparent surface which mounts between theuser and the display's viewing surface have several drawbacks. Forexample, the transparent surface, and other layers between the liquidcrystal material and the transparent surface may result in multiplereflections which decreases the display's contrast and produces glare.Moreover, adding an additional touch panel to the display increases themanufacturing expense of the display and increases the complexity of thedisplay. Also, the incorporation of the touch screen reduces the overallmanufacturing yield of the display.

Accordingly, what is desired is a touch screen that does notsignificantly decrease the contrast ratio, does not significantlyincrease the glare, does not significantly increase the expense of thedisplay, and does not significantly increase the complexity of thedisplay.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a cross sectional view of a traditional active matrix liquidcrystal display.

FIG. 2 is a schematic of the thin film transistor array.

FIG. 3 is a layout of the thin film transistor array of FIG. 2.

FIGS. 4A-4H is a set of steps suitable for constructing pixel electrodesand amorphous silicon thin-film transistors.

FIG. 5 illustrates pixel electrodes, color filters, and a black matrix.

FIG. 6 illustrates a schematic of the active matrix elements, pixelelectrode, photo TFT, readout TFT, and a black matrix.

FIG. 7 illustrates a pixel electrode, photo TFT, readout TFT, and ablack matrix.

FIG. 8 is a layout of the thin film transistor array of FIGS. 6 and 7.

FIG. 9 is a graph of the capacitive charge on the light sensitiveelements as a result of touching the display at high ambient lightingconditions.

FIG. 10 is a graph of the capacitive charge on the light sensitiveelements as a result of touching the display at low ambient lightingconditions.

FIG. 11 is a graph of the photo-currents in an amorphous silicon TFTarray.

FIG. 12 is a graph of the capacitive charge on the light sensitiveelements as a result of touching the display and providing light from alight pen.

FIG. 13 is an alternative layout of the pixel electrodes.

FIG. 14 illustrates a timing set for the layout of FIG. 13.

FIG. 15 illustrates a handheld device together with an optical wand.

FIG. 16 illustrates even/odd frame addressing.

FIG. 17 illustrates a front illuminated display.

FIG. 18 illustrates total internal reflections.

FIG. 19 illustrates a small amount of diffraction of the propagatinglight.

FIG. 20 illustrates significant diffraction as a result of a plasticpen.

FIG. 21 illustrates a shadow of a pointing device and a shadow withilluminated region of a pointing device.

FIG. 22 illustrates a modified black matrix arrangement.

FIG. 23 illustrates a light reflecting structure.

FIG. 24 illustrates a pen.

FIG. 25 illustrates a light guide and finger.

FIG. 26 illustrates a display with multiple sensor densities and opticalelements.

FIG. 27 illustrates a display with memory maintaining material.

FIG. 28 illustrates another pen

FIG. 29 illustrates another pen.

FIG. 30 illustrates another pen.

FIG. 31 illustrates another pen.

FIG. 32 illustrates another light guide.

FIG. 33 illustrates an image acquisition and processing technique.

FIG. 34 illustrates a display with an internal polarizer and no coverplate.

FIG. 35 illustrates ambient light inhibited light sensitive elements.

FIG. 36 illustrates a panel with light sensitive elements and a pressuresensor.

FIG. 37 illustrates a panel with light sensitive elements, lenses, andfilters.

DETAILED DESCRIPTION

Referring to FIG. 1, a liquid crystal display (LCD) 50 (indicated by abracket) comprises generally, a backlight 52 and a light valve 54(indicated by a bracket). Since liquid crystals do not emit light, mostLCD panels are backlit with fluorescent tubes, xenon flat lamp, orarrays of light-emitting diodes (LEDs) that are built into the sides orback of the panel. To disperse the light and obtain a more uniformintensity over the surface of the display, light from the backlight 52typically passes through a diffuser 56 before impinging on the lightvalve 54.

The transmittance of light from the backlight 52 to the eye of a viewer58, observing an image displayed on the front of the panel, iscontrolled by the light valve 54. The light valve 54 normally includes apair of polarizers 60 and 62 separated by a layer of liquid crystals 64contained in a cell gap between glass or plastic plates, and thepolarizers. Light from the backlight 52 impinging on the first polarizer62 comprises electromagnetic waves vibrating in a plurality of planes.Only that portion of the light vibrating in the plane of the opticalaxis of the polarizer passes through the polarizer. In an LCD lightvalve having a normally white configuration, the optical axes of thefirst 62 and second 60 polarizer are typically arranged at an angle sothat light passing through the first polarizer would normally be blockedfrom passing through the second polarizer in the series. However, theorientation of the translucent crystals in the layer of liquid crystals64 can be locally controlled to either “twist” the vibratory plane ofthe light into alignment with the optical axes of the polarizer,permitting light to pass through the light valve creating a brightpicture element or pixel, or out of alignment with the optical axis ofone of the polarizers, attenuating the light and creating a darker areaof the screen or pixel.

The surfaces of a first glass substrate 61 and a second glass substrate63 form the walls of the cell gap and are buffed to produce microscopicgrooves to physically align the molecules of liquid crystal 64immediately adjacent to the walls. Molecular forces cause adjacentliquid crystal molecules to attempt to align with their neighbors withthe result that the orientation of the molecules in the column ofmolecules spanning the cell gap twist over the length of the column.Likewise, the plane of vibration of light transiting the column ofmolecules will be “twisted” from the optical axis of the first polarizer62 to a plane determined by the orientation of the liquid crystals atthe opposite wall of the cell gap. If the wall of the cell gap is buffedto align adjacent crystals with the optical axis of the secondpolarizer, light from the backlight 52 can pass through the series ofpolarizers 60 and 62 to produce a lighted area of the display whenviewed from the front of the panel (a “normally white” LCD).

To darken a pixel and create an image, a voltage, typically controlledby a thin film transistor, is applied to an electrode in an array oftransparent electrodes deposited on the walls of the cell gap. Theliquid crystal molecules adjacent to the electrode are attracted by thefield produced by the voltage and rotate to align with the field. As themolecules of liquid crystal are rotated by the electric field, thecolumn of crystals is “untwisted,” and the optical axes of the crystalsadjacent to the cell wall are rotated progressively out of alignmentwith the optical axis of the corresponding polarizer progressivelyreducing the local transmittance of the light valve 54 and attenuatingthe luminance of the corresponding pixel. In other words, in a normallywhite twisted nematic device there are generally two modes of operation,one without a voltage applied to the molecules and one with a voltageapplied to the molecules. With a voltage applied (e.g., driven mode) tothe molecules the molecules rotate their polarization axis which resultsin inhibiting the passage of light to the viewer. Similarly, without avoltage applied (e.g., non-driven mode) the polarization axis is notrotated so that the passage of light is not inhibited to the viewer.

Conversely, the polarizers and buffing of the light valve can bearranged to produce a “normally black” LCD having pixels that are dark(light is blocked) when the electrodes are not energized and light whenthe electrodes are energized. Color LCD displays are created by varyingthe intensity of transmitted light for each of a plurality of primarycolor (typically, red, green, and blue) sub-pixels that make up adisplayed pixel.

The aforementioned example was described with respect to a twistednematic device. However, this description is only an example and otherdevices may likewise be used, including but not limited to, multi-domainvertical alignment, patterned vertical alignment, in-plane switching,and super-twisted nematic type LCDs. In addition other devices, such asfor example, plasma displays, organic displays, active matrix organiclight emitting display, electroluminescent displays, liquid crystal onsilicon displays, reflective liquid crystal devices may likewise beused. For such displays the light emitting portion of the display, orportion of the display that permits the display of selected portions oflight may be considered to selectively cause the pixels to providelight.

For an active matrix LCD (AMLCD) the inner surface of the second glasssubstrate 63 is normally coated with a continuous electrode while thefirst glass substrate 61 is patterned into individual pixel electrodes.The continuous electrode may be constructed using a transparentelectrode, such as indium tin oxide. The first glass substrate 61 mayinclude thin film transistors (TFTs) which act as individual switchesfor each pixel electrode (or group of pixel electrodes) corresponding toa pixel (or group of pixels). The TFTs are addressed by a set ofmultiplexed electrodes running along the gaps between the pixelelectrodes. Alternatively the pixel electrodes may be on a differentlayer from the TFTs. A pixel is addressed by applying voltage (orcurrent) to a selected line, which switches the TFT on and allows chargefrom the data line to flow onto the rear pixel electrodes. Thecombination of voltages between the front electrode and the pixelelectrodes sets up a voltage across the pixels and turns the respectivepixels on. The thin-film transistors are typically constructed fromamorphous silicon, while other types of switching devices may likewisebe used, such as for example, metal-insulator-metal diode andpolysilicon thin-film transistors. The TFT array and pixel electrodesmay alternatively be on the top of the liquid crystal material. Also,the continuous electrode may be patterned or portions selectivelyselected, as desired. Also the light sensitive elements may likewise belocated on the top, or otherwise above, of the liquid crystal material,if desired.

Referring to FIG. 2, the active matrix layer may include a set of datalines and a set of select lines. Normally one data line is included foreach column of pixels across the display and one select line is includedfor each row of pixels down the display, thereby creating an array ofconductive lines. To load the data to the respective pixels indicatingwhich pixels should be illuminated, normally in a row-by-row manner, aset of voltages are imposed on the respective data lines 204 whichimposes a voltage on the sources 202 of latching transistors 200. Theselection of a respective select line 210, interconnected to the gates212 of the respective latching transistors 200, permits the voltageimposed on the sources 202 to be passed to the drain 214 of the latchingtransistors 200. The drains 214 of the latching transistors 200 areelectrically connected to respective pixel electrodes and arecapacitively coupled to a respective common line 221 through arespective Cst capacitor 218. In addition, a respective capacitanceexists between the pixel electrodes enclosing the liquid crystalmaterial, noted as capacitances Clc 222 (between the pixel electrodesand the common electrode on the color plate). The common line 221provides a voltage reference. In other words, the voltage data(representative of the image to be displayed) is loaded into the datalines for a row of latching transistors 200 and imposing a voltage onthe select line 210 latches that data into the holding capacitors andhence the pixel electrodes. Alternatively, the display may be operatedbased upon current levels.

Referring to FIG. 3, a schematic layout is shown of the active matrixlayer. The pixel electrodes 230 are generally grouped into a “single”effective pixel so that a corresponding set of pixel electrodes 230 maybe associated with respective color filters (e.g., red, green, blue).The latching transistors 200 interconnect the respective pixelelectrodes 230 with the data lines and the select line. The pixelelectrodes 230 may be interconnected to the common line 221 by thecapacitors Cst 218. The pixels may include any desirable shape, anynumber of sub-pixels, and any set of color filters.

Referring to FIG. 4, the active matrix layer may be constructed using anamorphous silicon thin-film transistor fabrication process. The stepsmay include gate metal deposition (FIG. 4A), a photolithography/etch(FIG. 4B), a gate insulator and amorphous silicon deposition (FIG. 4C),a photolithography/etch (FIG. 4D), a source/drain metal deposition (FIG.4E), a photolithography/etch (FIG. 4F), an ITO deposition (FIG. 4G), anda photolithography/etch (FIG. 4H). Other processes may likewise be used,as desired.

The present inventors considered different potential architectural touchpanel schemes to incorporate additional optical layers between thepolarizer on the front of the liquid crystal display and the front ofthe display. These additional layers include, for example, glass plates,wire grids, transparent electrodes, plastic plates, spacers, and othermaterials. In addition, the present inventors considered the additionallayers with different optical characteristics, such as for example,birefringence, non-birefringence, narrow range of wavelengths, widerange of wavelengths, etc. After an extensive analysis of differentpotential configurations of the touch screen portion added to thedisplay together with materials having different optical properties andfurther being applied to the different types of technologies (e.g.,mechanical switches, crossed beams of infrared light, acoustic surfacewaves, capacitance sensing, and resistive membranes), the presentinventors determined that an optimized touch screen is merely a tradeoffbetween different undesirable properties. Accordingly, the design of anoptimized touch screen is an ultimately unsolvable task. In contrast todesigning an improved touch screen, the present inventors came to therealization that modification of the structure of the active matrixliquid crystal device itself could provide an improved touch screencapability without all of the attendant drawbacks to the touch screenconfiguration located on the front of the display.

Referring to FIG. 5, with particular attention to the latchingtransistors of the pixel electrodes, a black matrix 240 is overlying thelatching transistors so that significant ambient light does not strikethe transistors. Color filters 242 may be located above the pixelelectrodes. Ambient light striking the latching transistors results indraining the charge imposed on the pixel electrodes through thetransistor. The discharge of the charge imposed on the pixel electrodesresults in a decrease in the operational characteristics of the display,frequently to the extent that the display is rendered effectivelyinoperative. With the realization that amorphous silicon transistors aresensitive to light incident thereon, the present inventors determinedthat such transistors within the active matrix layer may be used as abasis upon which to detect the existence of or non-existence of ambientlight incident thereon (e.g., relative values thereto).

Referring to FIG. 6, a modified active matrix layer may include aphoto-sensitive structure or elements. The preferred photo-sensitivestructure includes a photo-sensitive thin film transistor (photo TFT)interconnected to a readout thin film transistor (readout TFT). Acapacitor Cst2 may interconnect the common line to the transistors.Referring to FIG. 7, a black matrix may be in an overlying relationshipto the readout TFT. The black matrix is preferably an opaque material orotherwise the structure of the display selectively inhibiting thetransmission of light to selective portions of the active matrix layer.Preferably the black matrix is completely overlying the amorphoussilicon portion of the readout TFT, and at least partially overlying theamorphous silicon portion of the readout TFT. Preferably the blackmatrix is completely non-overlying the amorphous silicon portion of thephoto TFT, and at least partially non-overlying the amorphous siliconportion of the photo TFT. Overlying does not necessarily denote directcontact between the layers, but is intended to denote in the generalsense the stacked structure of materials. The black matrix is preferablyfabricated on a layer other than the active plate, such as the colorplate. The active plate is normally referred to as the plate supportingthe thin-film transistors. The location of the black matrix on the colorplate (or other non-active plate) results in limited additionalprocessing or otherwise modification of the fabrication of the activematrix. In some embodiments, the black matrix inhibits ambient lightfrom impacting the amorphous silicon portion of the readout TFT to anextent greater than inhibiting ambient light from impacting theamorphous silicon portion of the photo TFT. A gate metal, or other lightinhibiting material, may inhibit the photo-sensitive elements from theback light.

Typically the photo-sensitive areas (channels of the transistors) aregenerally rectangular in shape, although other shapes may be used. Theopening in the black matrix is preferably wider (or longer) than thecorresponding channel area. The opening is preferably selected toaccommodate the desired numerical aperture for the light sensitiveelements. In this manner the channel area and the opening in the blackmatrix are overlapping, with the opening extending in a first dimension(e.g., width) greater than the channel area and in a second dimension(e.g., length) less than the channel area. This alignment alleviates theneed for precise registration of the layers while ensuring reasonableoptical passage of light to the light sensitive element. Other relativesizes may likewise be used, as described.

As an example, the common line may be set at a negative voltagepotential, such as −10 volts. During the previous readout cycle, avoltage is imposed on the select line which causes the voltage on thereadout line to be coupled to the drain of the photo TFT and the drainof the readout TFT, which results in a voltage potential across Cst2.The voltage coupled to the drain of the photo TFT and the drain of thereadout TFT is approximately ground (e.g., zero volts) with thenon-inverting input of the operational amplifier connected to ground.The voltage imposed on the select line is removed so that the readoutTFT will turn “off”.

Under normal operational conditions ambient light from the front of thedisplay passes through the black matrix opening and strikes theamorphous silicon of the photo TFT. However, if a person touches thefront of the display in a region over the opening in the black matrix orotherwise inhibits the passage of light through the front of the displayin a region over the opening in the black matrix, then the photo TFTtransistor will be in an “off” state. If the photo TFT is “off” then thevoltage across the capacitor Cst2 will not significantly dischargethrough the photo TFT. Accordingly, the charge imposed across Cst2 willbe substantially unchanged. In essence, the voltage imposed across Cst2will remain substantially unchanged if the ambient light is inhibitedfrom striking the photo TFT.

To determine the voltage across the capacitor Cst2, a voltage is imposedon the select line which causes the gate of the readout TFT tointerconnect the imposed voltage on Cst2 to the readout line. If thevoltage imposed on the readout line as a result of activating thereadout TFT is substantially unchanged, then the output of theoperational amplifier will be substantially unchanged (e.g., zero). Inthis manner, the system is able to determine whether the light to thedevice has been inhibited, in which case the system will determine thatthe screen has been touched at the corresponding portion of the displaywith the photo TFT.

During the readout cycle, the voltage imposed on the select line causesthe voltage on the respective drain of the photo TFT and the drain ofthe readout TFT to be coupled to the respective readout line, whichresults in resetting the voltage potential across Cst2. The voltagecoupled to the drain of the photo TFT and the drain of the readout TFTis approximately ground (e.g., zero volts) with the non-inverting inputof the operational amplifier connected to ground. The voltage imposed onthe select line is removed so that the readout TFT will turn “off”. Inthis manner, the act of reading the voltage simultaneously acts to resetthe voltage potential for the next cycle.

Under normal operational conditions ambient light from the front of thedisplay passes through the black matrix and strikes the amorphoussilicon of the photo TFT. If a person does not touch the front of thedisplay in a region over the opening in the black matrix or otherwiseinhibits the passage of light through the front of the display in aregion over the opening in the black matrix, then the photo TFTtransistor will be in an “on” state. If the photo TFT is “on” then thevoltage across the capacitor Cst2 will significantly discharge throughthe photo TFT, which is coupled to the common line. In essence thevoltage imposed across Cst2 will decrease toward the common voltage.Accordingly, the charge imposed across Cst2 will be substantiallychanged in the presence of ambient light. Moreover, there is asubstantial difference in the voltage potential across the holdcapacitor when the light is not inhibited versus when the light isinhibited.

Similarly, to determine the voltage across the capacitor Cst2, a voltageis imposed on the select line which causes the gate of the readout TFTto interconnect the imposed voltage to the readout line. If the voltageimposed on the readout line as a result of activating the readout TFT issubstantially changed or otherwise results in an injection of current,then the output of the operational amplifier will be substantiallynon-zero. The output voltage of the operational amplifier isproportional or otherwise associated with the charge on the capacitorCst2. In this manner, the system is able to determine whether the lightto the device has been uninhibited, in which case the system willdetermine that the screen has not been touched at the correspondingportion of the display with the photo TFT.

Referring to FIG. 8, a layout of the active matrix layer may include thephoto TFT, the capacitor Cst2, and the readout TFT in a region betweenthe pixel electrodes. Light sensitive elements are preferably includedat selected intervals within the active matrix layer. In this manner,the device may include touch panel sensitivity without the need foradditional touch panel layers attached to the front of the display. Inaddition, the additional photo TFT, readout TFT, and capacitor may befabricated together with the remainder of the active matrix layer,without the need for specialized processing. Moreover, the complexity ofthe fabrication process is only slightly increased so that the resultingmanufacturing yield will remain substantially unchanged. It is to beunderstood that other light sensitive elements may likewise be used. Inaddition, it is to be understood that other light sensitive electricalarchitectures may likewise be used.

Referring to FIG. 11, a graph of the photo-currents within amorphoussilicon TFTs is illustrated. Line 300 illustrates a dark ambientenvironment with the gate connected to the source of the photo TFT. Itwill be noted that the leakage currents are low and relatively stableover a range of voltages. Line 302 illustrates a dark ambientenvironment with a floating gate of the photo TFT. It will be noted thatthe leakage currents are generally low and relatively unstable over arange of voltages (significant slope). Line 304 illustrates a lowambient environment with the gate connected to the source of the photoTFT. It will be noted that the leakage currents are three orders ofmagnitude higher than the corresponding dark ambient conditions andrelatively stable over a range of voltages. Line 306 illustrates a lowambient environment with a floating gate of the photo TFT. It will benoted that the leakage currents are generally three orders of magnitudehigher and relatively unstable over a range of voltages (significantslope). Line 308 illustrates a high ambient environment with the gateconnected to the source of the photo TFT. It will be noted that theleakage currents are 4.5 orders of magnitude higher than thecorresponding dark ambient conditions and relatively stable over a rangeof voltages. Line 310 illustrates a high ambient environment with afloating gate of the photo TFT. It will be noted that the leakagecurrents are generally 4.5 orders of magnitude higher and relativelyunstable over a range of voltages (significant slope). With thesignificant difference between the dark state, the low ambient state,and the high ambient state, together with the substantially flatresponses over a voltage range (source-drain voltage), the system mayreadily process the data in a confident manner, especially with the gateconnected to the source. It is to be understood that the gate mayalternatively be biased with a different voltage. In general, thearchitecture preferably permits the leakage currents to be within oneorder of magnitude over the central 50%, more preferably over thecentral 75%, of the voltage range used for displaying images.

Referring to FIG. 9, under high ambient lighting conditions the photoTFT will tend to completely discharge the Cst2 capacitor to the commonvoltage, perhaps with an offset voltage because of the photo TFT. Inthis manner, all of the photo TFTs across the display will tend todischarge to the same voltage level. Those regions with reduced ambientlighting conditions or otherwise where the user blocks ambient lightfrom reaching the display, the Cst2 capacitor will not fully discharge,as illustrated by the downward spike in the graph. The downward spike inthe graph provides location information related to the region of thedisplay that has been touched.

Referring to FIG. 10, under lower ambient lighting conditions the photoTFT will tend to partially discharge the Cst2 capacitor to the commonvoltage. In this manner, all of the photo TFTs across the display willtend to discharge to some intermediate voltage levels. Those regionswith further reduced ambient lighting conditions or otherwise where theuser blocks ambient light from reaching the display, the Cst2 capacitorwill discharge to a significantly less extent, as illustrated by thedownward spike in the graph. The downward spike in the graph provideslocation information related to the region of the display that has beentouched. As shown in FIGS. 9 and 10, the region or regions where theuser inhibits light from reaching the display may be determined aslocalized minimums. In other embodiments, depending on the circuittopology, the location(s) where the user inhibits light from reachingthe display may be determined as localized maximums or otherwise somemeasure from the additional components.

In the circuit topology illustrated, the value of the capacitor Cst2 maybe selected such that it is suitable for high ambient lightingconditions or low ambient lighting conditions. For low ambient lightingconditions, a smaller capacitance may be selected so that the device ismore sensitive to changes in light. For high ambient lightingconditions, a larger capacitance may be selected so that the device isless sensitive to changes in light. In addition, the dimensions of thephototransistor may be selected to change the photo-leakage current.Also, one set of light sensitive elements (e.g., the photo TFT and thecapacitance) within the display may be optimized for low ambientlighting conditions while another set of light sensitive elements (e.g.,the photo TFT and the capacitance) within the display may be optimizedfor high ambient lighting conditions. Typically, the data from lightsensitive elements for low ambient conditions and the data from lightsensitive elements for high ambient conditions are separately processed,and the suitable set of data is selected. In this manner, the samedisplay device may be used for high and low ambient lighting conditions.In addition, multiple levels of sensitivity may be provided. It is to beunderstood that a single architecture may be provided with a wide rangeof sensitivity from low to high ambient lighting conditions. Inaddition, any suitable alternative architecture may be used for sensingthe decrease and/or increase in ambient light.

Another structure that may be included is selecting the value of thecapacitance so that under normal ambient lighting conditions the chargeon the capacitor only partially discharges. With a structure where thecapacitive charge only partially discharges, the present inventorsdetermined that an optical pointing device, such as a light wand orlaser pointer, might be used to point at the display to furtherdischarge particular regions of the display. In this manner, the regionof the display that the optical pointing device remains pointed at maybe detected as local maximums (or otherwise). In addition, those regionsof the display where light is inhibited will appear as local minimums(or otherwise). This provides the capability of detecting not only theabsence of light (e.g., touching the panel) but likewise those regionsof the display that have increased light incident thereon. Referring toFIG. 12, a graph illustrates local minimums (upward peaks) from addedlight and local maximums (downward peaks) from a lack of light. Inaddition, one set of light sensitive elements (e.g., the photo TFT andthe capacitance) within the display may be optimized for ambientlighting conditions to detect the absence of light while another set oflight sensitive elements (e.g., the photo TFT and the capacitance)within the display may be optimized for ambient lighting conditions todetect the additional light imposed thereon.

A switch associated with the display may be provided to select among aplurality of different sets of light sensitive elements. For example,one of the switches may select between low, medium, and high ambientlighting conditions. For example, another switch may select between atouch sensitive operation (absence of light) and an optical pointingdevice (addition of light). In addition, the optical pointing device maycommunicate to the display, such as through a wire or wirelessconnection, to automatically change to the optical sensing mode. A lightsensor (external photo-sensor to the light sensitive elements in theactive layer) and/or one or more of the light sensitive elements may beused to sense the ambient lighting conditions to select among differentsets of light sensitive elements. Also the sensor and/or one or morelight sensitive elements may be used to select, (1) to sense the absenceof light, (2) select to sense the addition of light, and/or (3) adjustthe sensing levels of the electronics.

In some embodiments the corresponding color filters for (e.g., above)some or all of the light sensitive elements may be omitted or replacedby a clear (or substantially clear) material. In this manner the lightreaching some of the light sensitive elements will not be filtered by acolor filter. This permits those light sensitive elements to sense agreater dynamic range or a different part of the dynamic range thanthose receiving filtered light.

It is noted that the teachings herein are likewise applicable totransmissive active matrix liquid crystal devices, reflective activematrix liquid crystal devices, transflective active matrix liquidcrystal devices, etc. In addition, the light sensitive elements maylikewise be provided within a passive liquid crystal display. Thesensing devices may be, for example, photo resistors and photo diodes.

Alternatively, light sensitive elements may be provided between the rearpolarizing element and the active matrix layer. In this arrangement, thelight sensitive elements are preferably fabricated on the polarizer, orotherwise a film attached to the polarizer. In addition, the lightsensitive elements may be provided on a thin glass plate between thepolarizer and the liquid crystal material. In addition, the black matrixor otherwise light inhibiting material is preferably arranged so as toinhibit ambient light from striking the readout TFT while free frominhibiting light from striking the photo TFT. Moreover, preferably alight blocking material is provided between the photo TFT and/or thereadout TFT and the backlight, such as gate metal, if provided, toinhibit the light from the backlight from reaching the photo TFT and/orthe readout TFT.

Alternatively, light sensitive elements may be provided between thefront polarizing element and the liquid crystal material. In thisarrangement, the light sensitive elements are preferably fabricated onthe polarizer, or otherwise a film attached to the polarizer. Inaddition, the light sensitive elements may be provided on a thin glassplate between the polarizer and the liquid crystal material. The lightsensitive elements may likewise be fabricated within the front electrodelayer by patterning the front electrode layer and including suitablefabrication techniques. In addition, a black matrix or otherwise lightinhibiting material is preferably arranged so as to inhibit ambientlight from striking the readout TFT while free from inhibiting lightfrom striking the photo TFT. Moreover, preferably a light blockingmaterial is provided between the photo TFT and/or the readout TFT andthe backlight, if provided, to inhibit the light from the backlight fromreaching the photo TFT and/or the readout TFT.

Alternatively, light sensitive elements may be provided between thefront of the display and the rear of the display, normally fabricated onone of the layers therein or fabricated on a separate layer providedwithin the stack of layers within the display. In the case of a liquidcrystal device with a backlight the light sensitive elements arepreferably provided between the front of the display and the backlightmaterial. The position of the light sensitive elements are preferablybetween (or at least partially) the pixel electrodes, when viewed from aplan view of the display. This may be particularly useful for reflectivedisplays where the pixel electrodes are opaque. In addition forreflective displays, any reflective conductive electrodes should bearranged so that they do not significantly inhibit light from reachingthe light sensitive elements. In this arrangement, the light sensitiveelements are preferably fabricated on one or more of the layers, orotherwise a plate attached to one or more of the layers. In addition, ablack matrix or otherwise light inhibiting material is preferablyarranged so as to inhibit ambient light from striking the readout TFTwhile free from inhibiting light from striking the photo TFT. Moreover,preferably a light blocking material is provided between the photo TFTand/or the readout TFT and the backlight, if provided, to inhibit thelight from the backlight from reaching the photo TFT and/or the readoutTFT.

In many applications it is desirable to modify the intensity of thebacklight for different lighting conditions. For example, in darkambient lighting conditions it may be beneficial to have a dimbacklight. In contrast, in bright ambient lighting conditions it may bebeneficial to have a bright backlight. The integrated light sensitiveelements within the display stack may be used as a measure of theambient lighting conditions as long as the elements are not insaturation to control the intensity of the backlight without the needfor an additional external photo-sensor. One light sensitive element maybe used, or a plurality of light sensitive element may be used togetherwith additional processing, such as averaging.

In one embodiment, the readout line may be included in a periodic mannerwithin the display sufficient to generally identify the location of the“touch”. For example the readout line may be periodically added at each30^(th) column. Spacing the readout lines at a significant number ofpixels apart results in a display that nearly maintains its previousbrightness because most of the pixel electrodes have an unchanged size.However, after considerable testing it was determined that such periodicspacing results in a noticeable non-uniform gray scale because ofdifferences in the size of the active region of the pixel electrodes.One potential resolution of the non-uniform gray scale is to modify theframe data in a manner consistent with the non-uniformity, such asincreasing the gray level of the pixel electrodes with a reduced size orotherwise reducing the gray levels of the non-reduced size pixelelectrodes, or a combination thereof. While a potential resolution, thisrequires additional data processing which increases the computationalcomplexity of the system.

A more desirable resolution of the non-uniform gray scale is to modifythe display to include a readout line at every fourth pixel, orotherwise in a manner consistent with the pixel electrode pattern of thedisplay (red pixel, green pixel, blue pixel). Alternatively, a readoutline is included at least every 12^(th) pixel (36 pixel electrodes of ared, blue, green arrangement), more preferably at least every 9^(th)pixel (27 pixel electrodes of a red, blue, green arrangement), even morepreferably at least every 6^(th) pixel (18 pixel electrodes of a red,blue, green arrangement or 24 pixel electrodes of a red, blue, greenarrangement), and most preferably at least every 3^(rd) pixel (3 pixelelectrodes of a red, blue, green arrangement). The readout lines arepreferably included for at least a pattern of four times the spacingbetween readout lines (e.g., 12 ^(th) pixel times 4 equals 48 pixels,9^(th) pixel times 4 equals 36 pixels). More preferably the pattern ofreadout lines is included over a majority of the display. The resultingdisplay may include more readout lines than are necessary to accuratelydetermine the location of the “touch”. To reduce the computationalcomplexity of the display, a selection of the readout lines may be freefrom interconnection or otherwise not operationally interconnected withreadout electronics. In addition, to further reduce the computationalcomplexity of the display and to increase the size of the pixelelectrodes, the readout lines not operationally interconnected withreadout electronics may likewise be free from an associated lightsensitive element. In other words, additional non-operational readoutlines may be included within the display to provide a gray scale displaywith increased uniformity. In an alternative embodiment, one or more ofthe non-operational readout lines may be replaced with spaces. In thismanner, the gray scale display may include increased uniformity, albeitwith additional spaces within the pixel electrode matrix.

The present inventors considered the selection of potential pixelelectrodes and came to the realization that the electrode correspondingto “blue” light does not contribute to the overall white transmission tothe extent that the “green” or “red” electrodes. Accordingly, the systemmay be designed in such a manner that the light sensitive elements areassociated with the “blue” electrodes to an extent greater than theirassociation with the “green” or “red” electrodes. In this manner, the“blue” pixel electrodes may be decreased in size to accommodate thelight sensitive elements while the white transmission remainssubstantially unchanged. Experiments have shown that reducing the sizeof the “blue” electrodes to approximately 85% of their original size,with the “green” and “red” electrodes remaining unchanged, results in areduction in the white transmission by only about 3 percent when sensorsare at every fourth pixel.

While such an additional set of non-operational readout lines providesfor increased uniform gray levels, the reduction of pixel aperturesresults in a reduction of brightness normally by at least 5 percent andpossibly as much as 15 percent depending on the resolution and layoutdesign rules employed. In addition, the manufacturing yield is decreasedbecause the readout line has a tendency to short to its neighboring dataline if the processing characteristics are not accurately controlled.For example, the data line and readout line may be approximately 6-10microns apart along a majority of their length.

Referring to FIG. 13, to increase the potential manufacturing yield andthe brightness of the display, the present inventors came to therealization that the readout of the photo-sensitive circuit and thewriting of data to the pixels may be combined on the same bus line, orotherwise a set of lines that are electrically interconnected to oneanother. To facilitate the use of the same bus line, a switch 418 mayselect between providing new data 420 to the selected pixels and readingdata 414 from the selected pixels. With the switch 418 set tointerconnect the new data 420 with the selected pixels, the data from aframe buffer or otherwise the video data stream may be provided to thepixels associated with one of the select lines. Multiple readoutcircuits may be used, or one or more multiplexed readout circuits may beused. For example, the new data 420 provided on data line 400 may be 4.5volts which is latched to the pixel electrode 402 and the photo TFT 404by imposing a suitable voltage on the select line 406. In this manner,the data voltage is latched to both the pixel electrode and acorresponding photo-sensitive circuit.

The display is illuminated in a traditional manner and the voltageimposed on the photo TFT 404 may be modified in accordance with thelight incident on the photo-sensitive circuit, as previously described.In the topology illustrated, the photo TFT 404 is normally a N-typetransistor which is reverse biased by setting the voltage on the commonline 408 to a voltage lower than an anticipated voltage on the photo TFT404, such as −10 or −15 volts. The data for the current frame may bestored in a frame buffer for later usage. Prior to writing the data foranother frame, such as the next frame, the data (e.g., voltage) on thereadout TFT 410 is read out. The switch 418 changes between the new data420 to the readout line 414 interconnected to the charge readoutamplifier 412. The select line 406 is again selected to couple theremaining voltage on the photo TFT 404 through the readout TFT 410 tothe data line 400. The coupled voltage (or current) to the data line 400is provided as an input to the charge readout amplifier 412 which iscompared against the corresponding data from the previous frame 422,namely, the voltage originally imposed on the photo TFT 404. Thedifference between the readout line 414 and the data from the previousframe 422 provides an output to the amplifier 412. The output of theamplifier 412 is provided to the processor. The greater the drain of thephoto TFT 404, normally as a result of sensing light, results in agreater output of the amplifier 412. Referring to FIG. 14, an exemplarytiming for the writing and readout on the shared data line 400 isillustrated.

At low ambient lighting conditions and at dark lighting conditions, theintegrated optical touch panel is not expected to operate well to thetouch of the finger because there will be an insufficient (or none)difference between the signals from the surrounding area and the touchedarea. To alleviate the inability to effectively sense at the low anddark ambient lighting conditions a light pen or laser pointer may beused (e.g., light source), as previously described. The light source maybe operably interconnected to the display such as by a wire or wirelesscommunication link. With the light source operably interconnected to thedisplay the intensity of the light source may be controlled, at least inpart, by feedback from the photo-sensitive elements or otherwise thedisplay, as illustrated in FIG. 15. When the display determines thatsufficient ambient light exists, such as ambient light exceeding athreshold value, the light source is turned “off”. In this manner,touching the light source against the display results in the same effectas touching a finger against the display, namely, impeding ambient lightfrom striking the display. When the display determines that insufficientambient light exists, such as ambient light failing to exceed athreshold value, the light source is turned “on”. In this manner,touching or otherwise directing the light from the light source againstthe display results in a localized increase in the received lightrelative to the ambient light level. This permits the display to beoperated in dark ambient lighting conditions or by feedback from thedisplay. In addition, the intensity of the light from the light sourcemay be varied, such as step-wise, linearly, non-linearly, orcontinuously, depending upon the ambient lighting conditions.Alternatively, the light source may include its own ambient lightdetector so that feedback from the display is unnecessary and likewisecommunication between the light source and the display may beunnecessary. Alternatively, the light pen may activate to emit lightupon sufficient pressure with the display and thus be deactivated so asto not emit light when no or insufficient pressure exists.

While using light from an external light source, while beneficial, itmay still be difficult to accurately detect the location of theadditional light because of background noise within the system andvariable lighting conditions. The present inventors considered thissituation and determined that by providing light during differentframes, such as odd frames or even frames, or odd fields or even fields,or every third frame, or during selected frames, a more defineddifferential signal between the frames indicates the “touch” location.In essence, the light may be turned on and off in some manner, such asblinking at a rate synchronized with the display line scanning orframes. An exemplary timing for an odd/even frame arrangement is shownin FIG. 16. In addition, the illumination of some types of displaysinvolves scanning the display in a row-by-row manner. In such a case,the differential signal may be improved by modifying the timing of thelight pulses in accordance with the timing of the gate pulse (e.g.,scanning) for the respective pixel electrodes. For example, in atop-down scanning display the light pulse should be earlier when thelight source is directed toward the top of the display as opposed to thebottom of the display. The synchronization may be based upon feedbackfrom the display, if desired.

In one embodiment, the light source may blink at a rate synchronizedwith the display line scanning. For example, the light source may usethe same driver source as the image pixel electrodes. In anotherembodiment the use of sequential (or otherwise) frames may be subtractedfrom one another which results in significant difference between signaland ambient conditions. Preferably, the light sensitive elements have adynamic range greater than 2 decades, and more preferably a dynamicrange greater than 4 decades. If desired, the system may use twosequential fields of scanning (all lines) subtracted from the next twofields of scanning (all lines) so that all the lines of the display areused.

Another technique for effective operation of the display in dark or lowlevel ambient conditions is using a pen or other device with a lightreflecting surface that is proximate (touching or near touching) thedisplay when interacting with the display. The light from the backlighttransmitted through the panel is then reflected back into thephoto-sensitive element and the readout signal will be greater at thetouch location than the surrounding area. Preferably, the touch targetregion of the display is at least partially transparent so that thereflected light will reach the light sensitive elements.

Referring to FIG. 17, another type of reflective liquid crystal display,typically used on handheld computing devices, involves incorporating alight guide in front of the liquid crystal material, which is normally aglass plate or clear plastic material. Normally, the light guide isconstructed from transparent material having an index of refractionbetween 1.4 and 1.6, more typically between 1.45 and 1.50, and sometimesof materials having an index of refraction of 1.46. The light guide mayfurther include anti-glare and anti-reflection coatings. The light guideis frequently illuminated with a light source, frequently disposed tothe side of the light guide. The light source may be any suitabledevice, such as for example, a cold cathode fluorescent lamp, anincandescent lamp, and a light emitting diode. To improve the lightcollection a reflector may be included behind the lamp to reflect lightthat is emitted away from the light guide, and to re-direct the lightinto the light guide. The light propagating within the light guidebounces between the two surfaces by total internal reflections. Thetotal internal reflections will occur for angles that are above thecritical angle, measured relative to the normal to the surfaces, asillustrated in FIG. 18. To a first order approximation, the criticalangle β may be defined by Sin(β)=1/n where n is the index of refractionof the light guide. Since the surfaces of the light guide are notperfectly smooth there will be some dispersion of the light, whichcauses some illumination of the display, as shown in FIG. 19.

The present inventors came to the realization that the critical angleand the disruption of the total internal reflections may be modified insuch a manner as to provide a localized increase in the diffusion oflight. Referring to FIG. 20, one suitable technique for the localizeddiffusion of light involves using a plastic pen to touch the front ofthe display. The internally reflected light coincident with the locationthat the pen touches the display will significantly diffuse and bedirected toward the photo sensitive elements within the display. Theplastic pen, or other object including the finger or the eraser of apencil, preferably has an index of refraction within 0.5, morepreferably within 0.25, of the index of refraction of the light guide.For example, the index of refraction of the light guide may be between1.2 and 1.9, and more preferably between 1.4 and 1.6. With the twoindexes of refraction sufficiently close to one another the disruptionof the internal reflections, and hence amount of light directed towardthe photo-sensitive elements, is increased. In addition, the plastic penpreferably has sufficient reflectivity of light (e.g., white tip, shinytip, reflective tip) as opposed to being non-reflective material, suchas for example, black felt.

Referring to FIG. 21, after further consideration the present inventorswere surprised to note that a white eraser a few millimeters away fromthe light guide results in a darkened region with generally consistentoptical properties while a white eraser in contact with the light guideresults in a darkened region with generally consistent opticalproperties together with a smaller illuminated region having greaterbrightness. In the preferred embodiment, the light sensitive elementsare positioned toward the front of the display in relation to the liquidcrystal material (or otherwise the light valve or electroluminescentmaterial) so that a clearer image may be obtained. It is to beunderstood that any suitable pointing device may be used. Theilluminated region has an illumination brighter in relation to theremainder of the darkened region. The illuminated region may be locatedby any suitable technique, such as for example, a center of gravitytechnique.

In some cases the display screen will have a relatively light coloredregion, such as white or tan, which is used as a virtual button foroperating software. Within this light colored region is typicallytextual information in relatively dark letters. In liquid crystaldisplay technology the light colored region is indicative of lightpassing through the liquid crystal material. Accordingly, if a pointinginstrument includes a generally reflective material the light passingthrough the display may be reflected back through the display. The lightreflected back through the display may be sensed by the light sensitiveelements.

After further consideration of the illuminated region the presentinventors came to the realization that when users use a “touch panel”display, there is a likelihood that the pointing device (or finger) may“hover” at a location above the display. Normally, during this hoveringthe user is not actually selecting any portion of the display, butrather still deciding where to select. In this manner, the illuminatedregion is beneficial because it provides a technique for thedetermination between when the user is simply “hovering” and the userhas actually touched (e.g., “touching”) the display.

In part, the sensitivity to hovering may be related to the lightsensitive elements being primarily sensitive to collimated light whichis inhibited by the finger or other element in proximity to the devicebecause of the alignment of the opening in the black matrix to the pixelelectrodes. To reduce the dependency to collimated light the blackmatrix may include central material aligned with the respective pixelelectrodes so that the light sensitive elements have an increasedsensitivity to non-collimated light (or otherwise non-perpendicular orotherwise angled incident light), as illustrated in FIG. 22. In someembodiments, the openings may be considered a non-continuous opening orotherwise the spatial opening for a particular pixel is non-continuous.

Another potential technique for the determination between “hovering” and“touching” is to temporally model the “shadow” region (e.g., lightimpeded region of the display). In one embodiment, when the user istypically touching the display then the end of the shadow will typicallyremain stationary for a period of time, which may be used as a basis, atleast in part, of “touching”. In another embodiment, the shadow willtypically enlarge as the pointing device approaches the display andshrinks as the pointing device recedes from the display, where thegeneral time between enlarging and receding may be used as a basis, atleast in part, of “touching”. In another embodiment, the shadow willtypically enlarge as the pointing device approaches the display andmaintain the same general size when the pointing device is touching thedisplay, where the general time where the shadow maintains the same sizemay be used as a basis, at least in part, of “touching”. In anotherembodiment, the shadow will typically darken as the pointing deviceapproaches the display and maintain the same shade when the pointingdevice is touching the display, where the general time where the shadowmaintains the same general shade may be used as a basis, at least inpart, of “touching”.

To further distinguish between the finger or other devices being closeto the display (or touching) or alternatively being spaced sufficientlyapart from the display, a light directing structure may be used. Onesuch light directing structure is shown in FIG. 23. The light directingstructure is preferably included around a portion of the periphery ofthe display and may reflect ambient light across the frontal region ofthe display. The reflected light then reflects off the finger or otherdevice thus increasing the light striking the light sensitive elementwhen the finger or other device is spaced sufficiently apart from thedisplay. The light reflecting off the finger or other device decreaseswhen the finger or other device is near the display because of theangular reflections of light. The differences in the reflected lightstriking the display may be used, at least in part, to detect thetouching of the display or otherwise inhibiting light to the display.

While attempting to consider implementation of such techniques on ahandheld device it came to the inventors' surprise that the displayportion of a handheld device has a refresh rate generally less than therefresh rate of the portion of the handwriting recognition portion ofthe display. The handheld portion of the display may use any recognitiontechnique, such as Palm OS™ based devices. The refresh rate of thedisplay is typically generally 60 hertz while the refresh rate of thehandwriting portion of the display is typically generally 100 hertz.Accordingly, the light-sensitive elements should be sampled at asampling rate corresponding with the refresh rate of the respectiveportion of the display.

The technique described with respect to FIG. 20 operates reasonably wellin dark ambient lighting conditions, low ambient lighting conditions,regular ambient lighting conditions, and high ambient lightingconditions. During regular and high ambient lighting conditions, thedisplay is alleviated of a dependency on the ambient lightingconditions. In addition, with such lighting the illumination point ismore pronounced and thus easier to extract. Unfortunately, during thedaytime the ambient light may be sufficiently high causing the detectionof the pointing device to be difficult. In addition, shades of theambient light may also interfere with the detection techniques.

The present inventors considered improving the robustness of thedetection techniques but came to the realization that with sufficient“noise” in the system the creation of such sufficiently robusttechniques would be difficult. As opposed to the traditional approach ofimproving the detection techniques, the present inventors came to therealization that by providing light to the light guide of a limitedselection of wavelengths and selectively filtering the wavelengths oflight within the display the difference between touched and un-touchedmay be increased. As an initial matter the light from the light sourceprovided to the light guide is modified, or otherwise filtered, toprovide a single color. Alternatively, the light source may providelight of a range of wavelengths, such as 600-700 nm, or 400-500 and530-580, or 630. Typically, the light provided to the light guide has arange of wavelengths (in any significant amount) less than white lightor otherwise the range of wavelengths of ambient light. Accordingly,with the light provided to the light guide having a limited color gamut(or reduced color spectrum) the touching of the pointing device on thedisplay results in the limited color gamut light being locally directedtoward the light-sensitive elements. With a limited color gamut lightbeing directed toward the display as a result of touching the lightguide (or otherwise touching the front of the display), a color filtermay be included between the light guide and the light-sensitive elementsto filter out at least a portion of the light not included within thelimited color gamut. In other words, the color filter reduces thetransmission of ambient light to an extent greater than the transmissionof light from the light source or otherwise within the light guide. Forexample, the ambient light may be considered as “white” light while thelight guide has primarily “red” light therein. A typical transmission ofa red color filter for ambient white light may be around 20%, while thesame color filter will transmit about 85% of the red light. Preferablythe transmission of ambient light through the color filter is less than75% (greater than 25% attenuation) (or 60%, 50%, 40%, 30%) while thetransmission of the respective light within the light guide is greaterthan 25% (less than 25% attenuation) (or 40%, 50%, 60%, 70%), so that inthis manner there is sufficient attenuation of selected wavelengths ofthe ambient light with respect to the wavelengths of light within thelight guide to increase the ability to accurately detect the touching.

In another embodiment, the light source to the light guide may include aswitch or otherwise automatic modification to “white” light whenoperated in low ambient lighting conditions. In this manner, the displaymay be more effectively viewed at low ambient lighting conditions.

In another embodiment, the present inventors determined that if thelight source providing light to the display was modulated in somefashion an improvement in signal detection may be achieved. For example,a pointing device with a light source associated therewith may modulatethe light source in accordance with the frame rate of the display. Witha frame rate of 60 hertz the pointing device may for example modulatethe light source at a rate of 30 hertz, 20 hertz, 10 hertz, etc. whichresults in additional light periodically being sensed by the lightsensitive elements. Preferably, the light source is modulated(“blinked”) at a rate synchronized with the display line scanning, anduses the same raw drivers as the image thin-film transistors. Theresulting data may be processed in a variety of different ways.

In one embodiment, the signals from the light sensitive elements areused, as captured. The resulting improvement in signal to backgroundratio is related to the pulse length of the light relative to the frametime. This provides some additional improvement in signal detectionbetween the light generated by the pointing device relative to theambient light (which is constant in time).

In another embodiment, multiple frames are compared against one anotherto detect the presence and absence of the additional light resultingfrom the modulation. In the case of subsequent frames (sequential ornon-sequential), one without additional light and one with additionallight, the data from the light sensitive elements may be subtracted fromone another. The improvement in the signal to background ratio isrelated to the periodic absence of the additional light. In addition,this processing technique is especially suitable for low ambientlighting and high ambient lighting conditions. Preferably the dynamicrange of the sensors is at least 4 decades, and two sequential frameswith additional light and two sequential frames without additional lightare used so that all of the scanning lines are encompassed. After thesystem may discharge the storage capacitor associated with the lightsensitive element as a result of sensing ambient light it resets thevoltage on the storage capacitor together with the readout operation.Since the first line will start at time zero and take a frame time, thelast line will be charged after almost a frame time and will takeanother frame time to discharge. Therefore, the system should preferablyuse two frames with additional illumination and then two frames withoutadditional illumination.

While the light sensitive elements may be used to determine “touch” andthe location of the “touch”, it is sometimes problematic to distinguishbetween “hovering” and “touch”. To assist in the determination of actualtouching of the display a pressure based mechanism may be used. Onepressure based mechanism may include pressure sensitive tape between apair of layers of the display or between the display and a support forthe display. Another pressure based mechanism may include an electricalor magnetic sensor operably connected to the display. In either case,the pressure based mechanism provides a signal to the displayelectronics indicating the sensing of pressure (e.g., touch) oralternatively the absence of pressure (e.g., non-touch).

Referring to FIG. 24, one configuration of an elongate light emittingdevice includes an infra-red light emitting diode that periodically orcontinuously emits an infra-red beam. The infra-red beam is transmittedfrom the light emitting device and reflects off the display. When thelight pen is spaced sufficiently far from the display the reflectedinfra-red beam will not strike the light pen. When the light pen isspaced sufficiently close to the display the reflected infra-red beamwill strike the light pen and is sensed by an infra-red sensor withinthe light pen. Infra-red light is preferred, while any suitablewavelength may be used that the light sensitive elements of the displayare generally insensitive to. When the infra-red sensor senses thereflected infra-red light the visible light emitting diode is turned onto illuminate the pixel. The visible light emitting diode preferablyprovides a wavelength that the light sensitive elements of the displayare sensitive to. After a predetermined duration or otherwise while theinfra-red light is not being sensed by the infra-red sensor the visiblelight emitting diode is turned off. In this manner, battery power withinthe light pen is conserved. In addition, the edge or shape of thevisible light from the visible light emitting diode may be used todetermine the spacing between the light pen and the display. Also, thebeam from the visible light emitting diode may be varied based upon thesignal sensed by the infra-red sensor.

Upon reconsidering the display with a light guide, as illustrated inFIG. 17, the present inventors realized that those portions of the lightguide that are in contact with the high portion of the user'sfingerprints will tend to diffuse and scatter light toward the lightsensitive elements, as illustrated in FIG. 25. Those portions of thelight guide that are not in contact with the high portion of the user'sfingerprints (i.e., valleys) will not tend to diffuse and scatter lighttoward the light sensitive elements. Depending on the thickness of thelight guide and liquid crystal material together with the density of thelight sensitive elements, the details observable in the fingerprint willvary. To increase the ability to detect the fingerprint, the display maybe designed with multiple densities of light sensitive elements. Forexample, in the region where the fingerprint is normally located thedensity of the light sensitive elements may be increased such asincluding a light sensitive element at every sub-pixel. In this manner,the display includes multiple densities of light sensitive elements.Moreover, the display may likewise be used for sensing other items, suchas for example, bar codes.

In some cases there may be excessive parallax which causes a smearedimage to be detected. In addition, oil and sweat on the fingers may tendto reduce the contrast between the high and low point of the user'sfingerprint. Referring to FIG. 26, a separate sensor structure may beincluded within the display. The sensor structure may include a lensbetween the light guide and the light sensitive elements. The lens maybe any suitable lens structure, such as for example, a small focus lensor a SELFOC lens (variable index of refraction fiber optics). The colorfilters in the fingerprint sensing region may be omitted, if desired. Insome cases, the color filter and the lenses may be omitted on a portionof the display, such as extending the active plate edge beyond theviewing area. In that region preferably a thin film protects the lightsensitive elements and the finger is in close proximity to the lightsensitive elements so that the resolution is maintained.

To increase the ability of the light sensitive elements to detect thedetails that may be present within a fingerprint it is desirable to havea greater density of light sensitive elements in the region where thefingerprint is to be sensed. In this manner the display may include two(or more) different densities of light sensitive elements across thedisplay. Alternatively, the light guide and lens may be omitted and ahigh density of light sensitive elements included for fingerprintsensing. It is to be understood that other items may likewise be sensedwith the high density light sensitive elements.

The light sensitive elements will tend to observe very high ambientlighting conditions when the finger is not present rendering theirability to detect high contrast images difficult. To increase theeffective sensitivity of the light sensitive elements a color filter maybe provided in an overlying relationship to the light sensitiveelements. In addition, the light source may be selected in relation tothe color filter. For example, a blue filter may be used together with ablue light source. In addition, the illumination may be modulated andsynchronized with the sensors. In this case, the light source may beilluminated with relatively short pulses together with the triggering ofthe sensing by the light sensitive elements. In the case where the lightis pulsed in relation to the frame rate it is preferably pulsed at halfthe frame rate. In this manner the light pulses will be ensured to besensed during different frames. Also, a light inhibiting material mayclosely surround the region where the finger is locate to reduce strayambient light, thus increasing contrast.

For the detection of fingerprints, sweat and oil from the fingers mayimpact the ability to accurate sense the fingerprint. Sweat is primarilywater with an index of refraction of approximately 1.3. Typical glasshas an index of refraction of approximately 1.5 which is sufficientlydifferent than 1.3, and accordingly sweat will not significantlynegatively impact image sensing. However, oils have an index ofrefraction of approximately 1.44 to 1.47 which is considerably closer to1.5, and accordingly oil will tend to significantly impact imagesensing. In order to improve the contrast, hence image sensing, theglass may be replaced with glass having a higher index of refraction orglass coated with a material having a higher index of refraction, suchas for example, 1.55 or more.

Attempting to record signatures may be problematic with many touchsensitive displays because the response time of the recording system isinadequate or otherwise the software has a slow sampling rate. However,the user normally prefers immediate feedback. Referring to FIG. 27 thedisplay may include a signature portion that includes a memorymaintaining material. The memory maintaining material may sense thewriting of the signature, such as by pressure exerted thereon or lightsensitive material. For example, the memory material may be a pair offlexible layers with fluid there between that is displaced while writingthe signature. After writing the name the user may press a button orotherwise touch another portion of the display indicating that thesignature is completed. Alternatively, the display may sense thesignature by a sufficient change in the optical properties of the memorymaintaining material. The display may then capture an image of thesignature. For example, the image of the signature may come from ambientlight passing through the memory maintaining material or otherwise fromlight reflected off the memory maintaining material, especially when thedisplay is in transmissive (“white”) mode. The signature may be clearedin any manner, such as electrical erasure or physical erasure by anysuitable mechanism. In this manner, the temporal limitations of writinga signature are reduced.

Alternatively the display may include a signature mode that captures theuser's signature over several frames. The signature may be captured on apredetermined region of the display or otherwise any portion of thedisplay. The system detects the decrease in ambient light over a seriesof frames as the signature is written. In this manner the “path” of thesignature may be determined and thereafter used in any suitable manner.

In many cases the user desires the tactile response of pen pressureagainst the display. In this manner, the user has greater comfort withthe pen and display for writing, drawing, or otherwise indicating aresponse. Referring to FIG. 28, the pen may include a movable opticalpath (e.g., a fiber optic bundle) with respect to the pen that extendsand retracts based upon pressure exerted on the display. A lightemitting diode provides a beam of light that is channeled through theoptical path. As it may be observed, when the optical path is in aretracted state that more light passes through the optical path thanwhen the optical path is in an extended state. In this manner, the lightsensitive elements may detect the intensity of the transmitted light anddetermine the pressure that is being exerted against the display by theuser. This capability is particular useful for pressure sensitiveapplications, such as Photoshop by Adobe.

Referring to FIG. 29 another pen pressure embodiment is illustrated. Acylindrical tubular tip portion is movable with respect to the pen thatextends and retracts based upon pressed exerted on the display. Withinthe cylindrical tubular tip portion is a lens. The lens focuses thelight emitted from a light emitting diode, which preferably ismaintained stationary with respect to the pen and/or moves with respectto the cylindrical tubular tip portion. Alternatively, the lightemitting diode may be connected to the tip portion of the pen. As it maybe observed, when the cylindrical tubular tip portion is in a retractedstate light may be more focused on the display (light sensitiveelements) than when the cylindrical tubular tip portion is in anextended state. The lens may be modified so that it operates in areversed manner. The focus of the beam may be detected in any suitablemanner by the light sensitive elements (e.g., size and/or intensity) todetermine the pressure that is being exerted against the display by theuser.

Referring to FIG. 30 another pen pressure embodiment is illustrated. Acylindrical tubular tip portion or optical light guide is movable withrespect to the pen that extends and retracts based upon pressure exertedon the display. As the cylindrical tubular tip portion moves withrespect to the pen the resistance of a variable resistive elementchanges. The variable resistive element is interconnected to a lightemitting diode which changes intensity based upon the change in thevariable resistance. The intensity of light sensed by the lightsensitive elements, or otherwise the change in intensity sensed by thelight sensitive elements, may be used to determine the pressure that isbeing exerted against the display by the user.

Referring to FIG. 31 another pen pressure embodiment is illustrated. Anoptional lens focuses a beam from a light emitting diode. When the penis farther from the display a larger spot size and/or intensity issensed by the light sensitive elements with respect to when the pen iscloser to the display. The intensity of the light and/or the size of thespot sensed by the light sensitive elements or otherwise the change inintensity and/or size may be used to determine the pressure that isbeing exerted against the display by the user.

Referring to FIG. 32, a modified light guide type structure is shown. Apolarizer may be included on the lower surface of the light guide. Inaddition, other polarizers may be included on the front and rearsurfaces of the liquid crystal display. The polarizer on the lowersurface of the light guide preferably matches (+/−5 degrees, +/−2degrees) the orientation of the polarizer on the front surface of theliquid crystal display. The polarizers may include anti-reflectioncoatings if desired. With the top two polarizers being included togetherwith proper alignment between them more of the light from the pen willtend to pass through to the liquid crystal display.

Referring to FIG. 33 an exemplary image processing technique isillustrated for processing the data from the display to determine thelocation of the touch. The display is initially calibrated by acalibration image module. For calibration, an image is obtained with thedisplay covered by a black cloth or otherwise blocked from receivingambient light. The black reference image may be referred to as I0. Thenan image may be obtained under normal uniform (e.g., without significantshadows or light spots) ambient lighting conditions and referred to asI1. A comparison between I0 and I1 may be performed to calibrate thedisplay.

During operation, the image is initially captured by a capture imagemodule. For example, a 60×60 sensor matrix may be captured. Optionally aset of consecutive frames may then be averaged by an average framemodule in order to reduce the noise in the signal, such as for examplefour frames.

An AGC module may perform an automatic gain control function in order toadjust for an offset in the value. One particular implementation may bedescribed by the following equation: Ie={(Is−I0)/(I1−I0)}. Ie is thesignal after equalization. “Is” is the sensor signal as captured orafter averaging. I0 is the stored sensor reading for the black referenceimage. I1 is the stored sensor reading for the bright (e.g., ambientlighting conditions) reference image. The equalization module uses I0 toadjust for the potential non-zero value at dark conditions. Inparticular, this adjustment is made by the calculation of Is-I0. Theresulting comparison (e.g., division) of the captured signal versus thestored bright reference signals adjusts the level of the signal.Moreover, the output of the equalization is a normalized signal with arange from 0 to 1 in the case that Is<I1. Ie may then be used to adjustthe gain of the output of the average frame module to captured imagevalue. The AGC module thus effectively corrects for dark levelnon-uniformity and for sensor gain non-uniformity.

A smoothing module may be used to average proximate values together tocompensate for non-uniformity in the characteristics of the display. Asuitable filter is an averaging of the 8 adjacent pixels to a centralpixel. Inherently the system provides relatively sharp edges from thesignals which may be used directly. However, with the capability in someembodiments of detecting positive and negative outputs it was determinedthat using an edge detection module is beneficial. A preferred edgedetection technique uses a 3×3 matrix, such as for example: {(−1 −1 −1)(−1 8 −1) (−1 −1 −1)}. The effect of the edge detection module is toenhance or otherwise determine those regions of the image that indicatean edge. Other edge detection techniques may likewise be used, such asfor example, a Sobel edge detection technique, a 1^(st) derivativetechnique, and a Robert's cross technique.

A threshold module may be used to set all values below a predeterminedthreshold value to zero, or otherwise indicate that they are not edgesor regions being touched. In the case that the system provides negativevalues the predetermined threshold value may be less than an absolutevalue. The use of threshold values assists in the discrimination betweenthe end of the finger touching the display and the shade from the handitself. If there are an insufficient number of pixels as a result of thethreshold module that are non-thresholded then the system returns to thestart.

If there are a sufficient number of pixels as a result of the thresholdmodule that are non-thresholded then the system determines the largestregion of non-thresholded values using a max location module. In thismanner, smaller regions of a few values may be removed so that thepredominant region of non-thresholded values may be determined. A centerof gravity module may be used to determine the center of the maximumregion from the max location module. An x-y coordinates module may beused to provide the x and y display coordinates and a plot cross modulemay be used to display a cross on the display at the x-y coordinates ofthe center of gravity. The cross module may provide data regarding theexistence of the “touch” and its location to the system and returncontrol back to the start.

Periodically the display may become scratched or otherwise a foreignobject will be stuck to the front of the display. In this case thedisplay will tend to provide false readings that the scratch or foreignobject is indicative of a touch. In order to reduce the effects ofscratches and foreign objects one or more bright reference images I1 maybe obtained over a period of time. The set of one or more brightreference images I1 may be averaged if an insubstantial differenceexists between the images. This reduces the likelihood that the displaywas touched during one or more of the image acquisitions. In the eventthat the images are substantially different then the images may bereacquired until an insubstantial difference exists.

When operated at low ambient lighting conditions it is difficult todetect the touching of the display. While under normal ambient lightingconditions it is desirable to adjust the output of the display inrelation to the black reference image (I0). However, it has beendetermined that under low ambient lighting conditions it is desirable toadjust the output of the display in relation to the bright referenceimage (I1). Accordingly, one particular implementation may be describedby the following equation: Ie={(Is-I1)/(I1-I0)}. The switching to nightmode (low ambient lighting conditions) may be automatic or in responseto a switch.

The typical construction of a liquid crystal panel is to locate a rearpolarizer between the backlight and the liquid crystal material. A frontpolarizer is located, typically with a perpendicular polarization axisto the rear polarizer, between the liquid crystal material and the frontof the device. A glass, or otherwise transparent, cover plate is locatedon top of the front polarizer and its supporting surface to protect thepolarizer from being damaged by scratching as a result of the usertouching the display. Unfortunately, the cover glass significantlyincreases the distance between the front of the display and the lightsensitive elements, which are normally located together with the pixelelectrodes, resulting in an increased difficulty in properly locatingthe position of the touch. Also, ambient light tends to be scattered bythe cover glass resulting in spurious light being sensed by the lightsensitive elements, also resulting in an increased difficulty inproperly locating the position of the touch.

To overcome such limitations, the present inventors came to therealization that the cover glass should be eliminated and the locationof the front polarizing element should be relocated to a position sothat its front surface is not exposed to the touch of the user. In thismanner, the distance between the front of the display and the lightsensitive elements is reduced, and the scattering which would haveresulted from the cover glass is eliminated. Accordingly, the coverageof the aperture of the light sensitive elements is improved.

One suitable structure for elimination of the cover glass and locatingthe front polarizing element is shown in FIG. 34. The structure includesa backlight 600, a glass with thin film transistors 602, a polarizer604, a polyimide layer 606, liquid crystal material 608, a polyimidelayer 610, a polarizer 612, and glass supporting the front electrodes614. As it may be observed, this structure locates the polarizer 612internal to the glass supporting the front electrodes 614. A similarstructure is disclosed in, Bobrov, et al., in a paper entitled, “5.2Manufacturing of a Thin-Film LCD”. The configuration of a touchsensitive display without including a cover glass is counterintuitive todesigners of liquid crystal displays because a touch panel device isconsidered to require a cover glass, while a non-touch panel devicetypically does not include the cover glass.

Another structure includes incorporating a hard coat layer on the uppersurface of the device. The hard coat layer is normally 200 nanometers orless in thickness. In addition, the glass (or otherwise) used to supportthe front electrodes is preferably 1.1 mm, and within the range of 1.3mm and 0.7 mm.

The present inventors observed that different light sensitive elements,such as transistors, tended to change their sensitivity andresponsiveness to light over time during the operation of the display,from location to location within the display, and from display todisplay. After consideration of this unexpected occurrence, the presentinventors speculated that the source of this effect is the dark currentof the transistors that may result from charges imposed on thetransistors from the environment, such as the glass, the liquid crystalmaterial, the temperature of the display, the circuitry, etc. The darkcurrent may be defined as that current which exists when the transistoris substantially free (or otherwise free) from sensing light. It waspreviously accepted that the transistors had no current when no lightwas striking the transistors and that the current increased as increasedlight reached the transistors.

To determine what the dark current is for the transistors of the displayor transistors located at different regions of the display in the eventthat the dark current has local variations, the present inventorsdetermined that additional transistors may be included within thedisplay that are substantially free (or otherwise free) from sensingambient light. The selected transistors may include a material coveringthe transistor to inhibit ambient light from reaching the selectedtransistors. Referring to FIG. 35, the upper row of sensors may be movedabove the openings in the black mask to block light from reaching thetransistors. In general, any of the existing sensors associated with thetraditional control circuitry of the display may be moved to a locationthat is under the black mask to block light from reaching thetransistors. This permits an existing row of sensors, or otherwise, tobe used if it is impracticable to modify the control circuitry to createa new row (or otherwise). Also, an additional row of sensors (orotherwise) may be added below the last row of exposed sensors (orotherwise), which may require a modification to the traditional controlcircuitry. The transistors may likewise be aligned in a column, ifdesired.

The transistors that are inhibited from sensing light may be used tosense the “dark current” present within the entire display, or differentregions of the display. Based upon the dark current sensed from theselight inhibited transistors, the display may calibrate the light sensingtransistors within an entire display or otherwise different regions ofthe display. In this manner, the output of the light sensing transistorsmay be calibrated, at least in part, based upon the other lightinhibited transistors, such as for example, to re-calibrate sensitivityand its “off” value.

While the light sensitive elements tend to provide good informationregarding the user touching the display, there are times thatdetermining whether the user touched the screen or otherwise just had afinger in the vicinity of the screen is difficult to discern. Such timesof difficult discernment tend to occur when highly collimated light isprovided to the display. In the case of highly collimated light, thefinger tends to block the light from reaching the display when it istouching the display and when it is merely in the vicinity of thedisplay. In the case of a highly scattered light source, the finger doesnot tend to block sufficient light from reaching the display when it ismerely in the vicinity of the display (e.g., light goes under the fingerat oblique angles), and tends to block the light from reaching thedisplay when it is touching the display (e.g., less space for light togo under the finger at oblique angles). Accordingly, it is difficult todiscern touch versus “hovering” with a highly collimated light source.

To alleviate concerns regarding whether the user touched the display oris merely “hovering” in the vicinity of the display, the presentinventors determined that a sensor should be incorporated within thedisplay to sense pressure (e.g., vibrations). The output of thevibration sensor is coupled to the processor so that it may bedetermined when the user touched the display, determine when the userdid not touch the display, or otherwise provide additional confidence tothe determination as to whether the user touched or did not touch thedisplay. Referring to FIG. 36, any suitable location for the sensor maybe used, such as for example, interconnected to the backside center ofthe chassis. Any suitable sensor may be used, such as for example,micro-machined accelerometers, piezoelectric devices such as PZT orPVDF, capacitive sensors, resistive sensors, inductive sensors, laservibrometers, and light emitting diode vibrometers. On way of selectingwhen a touch occurred is when the minimum of impact energy as a givenfrequency range has been reached. The sensor may also be used inconjunction with the light sensitive elements to determine the locationof the touch.

As it may be observed, the display may be used to identify the shape ofthe finger of a user, or otherwise the shape of something in pressingengagement, or otherwise in close proximity with the display, such asthe hand of a user. The shape of the user's finger or hand typicallyextends over many of the light sensitive elements and the shape isdefined by the edges of the inhibited light sensed by the lightsensitive elements. The shape sensed by the light sensitive elements maybe used to identify the user or otherwise identify the region that istouched.

In the case that the light reaching the display is primarily from apoint source is sufficiently spaced apart and generally aligned in adirection perpendicular to the display then the shape sensed by thelight sensitive elements tends to be relatively accurate. In addition,the shape sensed by the light sensitive elements tends to be relativelyaccurate, in the event that the light is highly diffused which generallyinhibits light all around the edges of the shape in a uniform manner.However, the combination of highly diffused light and a point sourcethat is not generally aligned in a direction perpendicular to thedisplay results in a sensed shape that is distorted. Further, if thefinger or hand of the user is not flat against the display, such ashaving a hand pressed against the display with raised fingers, theraised portions tend to result in sensing smeared and blurred edges.Accordingly, the edges may be difficult to accurately discern.

After consideration of such potential imaging problems, the presentinventors observed that the existing light sensitive elements areembedded in the display with a viewing angle defined by the thickness ofthe glass above them and the index of refraction of the glass. This typeof configuration results in a sensor system without a focal depth.Referring to FIG. 37, to provide a focal depth and permit the light tobe effectively focused, a set of micro-lenses may be located in front ofthe sensors. The lenses should have a focal length matching the locationof the sensors within the structure. A typical focal length is about 1mm to 1.5 mm. With such a configuration the sensors will get lightgenerally perpendicular to the display focused into the sensors.

While such lenses provide suitable focus of the light, it has beendetermined that it would be desirable to inhibit highly angled light(with respect to a normal to the display) from being sensed by thesensors, so as to improve the directionality of the light reaching thesensors. Improved directionality increases the sharpness of the edges ofwhat is being sensed and decreases the distortion. To improvedirectionality the display may include a suitable material thereon, suchas for example, Louver filters or fiber optic face plates.

Louver filters are commonly used as privacy filters consisting of filmswith alternate black thin vertical layers (opaque) and thicker clearlayers (clear), and behave in a manner similar to Venetian blinds.Accordingly, the Louver filters tend to significantly narrow the viewingangle in a single direction. In addition, two layers of Louver filers atright angles to another tend to significantly narrow the viewing anglein an opposing direction. In this manner, the undesirable effect of apoint source providing light in a non-perpendicular direction and highlydiffused light are reduced.

A fiber optic faceplate has a similar effect in the reduction of angularsensitivity. The fiber optics are selected with a sufficiently smallnumerical aperture. The aperture is typically defined by the index ofrefraction difference between the core of the fiber and the clad.

The filters or faceplate (or otherwise optically directional material)may be used in combination with the lenses, if desired. This permitseffective focus of light on the sensors while simultaneously selectingthe preferred light for such focusing to achieve accurate imaging.

All references cited herein are hereby incorporated by reference.

The terms and expressions which have been employed in the foregoingspecification are used therein as terms of description and not oflimitation, and there is no intention, in the use of such terms andexpressions, of excluding equivalents of the features shown anddescribed or portions thereof, it being recognized that the scope of theinvention is defined and limited only by the claims which follow.

What is claimed is:
 1. A system comprising: a display comprising a stackof layers configured to display an image; multiple elements included inthe stack of layers, wherein each of the multiple elements is configuredto sense light; a pressure sensor distinct from the multiple elementsand configured to sense a pressure applied to the display; and aprocessor configured to detect an object based on output from themultiple elements and output from the pressure sensor.
 2. The display ofclaim 1, wherein the processor is configured to determine that theobject has touched the display based on output from the multipleelements indicating a region of sensed light intensity below a thresholdlevel of sensed light intensity, and output from the pressure sensorexceeding a threshold level.
 3. The display of claim 1, wherein theprocessor is configured to determine that the object is hovering overthe display based on output from the multiple elements indicating aregion of sensed light intensity below a threshold level of sensed lightintensity, and output from the pressure sensor falling within athreshold level.
 4. The system of claim 1, wherein the pressure sensorcomprises pressure sensitive tape.
 5. The system of claim 4, wherein thepressure sensitive tape is located directly between a pair of layers ofthe display.
 6. The system of claim 4, wherein the pressure sensitivetape is located directly between the display and a support for thedisplay.
 7. The system of claim 1, wherein the pressure sensor comprisesan accelerometer.
 8. The system of claim 1, wherein the pressure sensorcomprises a piezoelectric device.
 9. The system of claim 1, wherein thepressure sensor comprises a capacitive sensor.
 10. The system of claim1, wherein the pressure sensor comprises a resistive sensor.
 11. Thesystem of claim 1, wherein the pressure sensor comprises an inductivesensor.
 12. The system of claim 1, wherein the pressure sensor comprisesa laser vibrometer.
 13. The system of claim 1, wherein the pressuresensor comprises a light emitting diode vibrometer.
 14. A displaycomprising: a stack of layers configured to display an image; multipleelements included in the stack of layers, wherein each of the multipleelements is configured to sense light; a pressure sensor distinct fromthe multiple elements and configured to sense a pressure applied to thedisplay; and a processor configured to detect an object based on outputfrom the multiple elements and output from the pressure sensor.
 15. Thedisplay of claim 14, wherein the processor is configured to determinethat the object has touched the display based on output from themultiple elements indicating a region of sensed light intensity below athreshold level of sensed light intensity, and output from the pressuresensor exceeding a threshold level.
 16. The display of claim 14, whereinthe processor is configured to determine that the object is hoveringover the display based on output from the multiple elements indicating aregion of sensed light intensity below a threshold level of sensed lightintensity, and output from the pressure sensor falling within athreshold level.
 17. The display of claim 14, wherein the pressuresensor comprises pressure sensitive tape.
 18. The display of claim 17,wherein the pressure sensitive tape is located directly between a pairof layers of the display.
 19. The display of claim 14, wherein thepressure sensor comprises an accelerometer.